Method and system for defining cut lines to generate a 3D fetal representation

ABSTRACT

A plurality of ultrasound frames of a fetus are acquired using an ultrasound scanner, which may be oriented arbitrarily with respect to the fetus during the acquisition. The ultrasound frames are processed against an artificial intelligence model to predict a different cut line on each of the ultrasound frames. Each cut line is predicted to be exterior to an image of the fetus appearing on the ultrasound frame. The different cut lines on the plurality of ultrasound frames are then used to identify ultrasound data in the image frames to generate a 3D representation of the fetus.

TECHNICAL FIELD

This disclosure relates to ultrasound imaging. In particular, it relatesto systems and methods for generating a three dimensional (3D) fetalrepresentation.

BACKGROUND

Ultrasound is a useful, non-invasive imaging technique capable ofproducing real time images of internal structures within tissue.Ultrasound imaging has an advantage over X-ray imaging in thatultrasound imaging does not involve ionizing radiation. Some mobileultrasound scanners, including app-based ultrasound scanners,communicate with an add-on device that can act as both as a display anda control device. Examples of these add-on devices are mobile phones,tablets, laptops or desktop computers.

When using some ultrasound scanners (whether mobile or not) forgenerating a 3D fetal representation, users are traditionally expectedto orientate the ultrasound scanner to provide an image of themidsaggital plane of the fetus. They are then expected to define aregion of interest and/or a cut line on a frame of this two dimensional(2D) ultrasound image. The cut line defines a portion of the ultrasoundframe that excludes the fetus, and which is removed for generating the3D representation. The same cut line is then used on multiple 2Dultrasound frame slices, which are subsequently acquired while holdingthe ultrasound scanner in a still position or as the ultrasound scannersweeps across the skin surface.

Some ultrasound scanning systems allow 4D ultrasound representations tobe obtained, in which a 3D representation is obtained repeatedly so thatthe fetus can be viewed in 3D in real time.

A skilled ultrasound operator is required, first to obtain themidsaggital plane, and then to define a suitable cut line on theultrasound frame. Furthermore, use of the same cut line for the multiple2D image slices may cause imperfections to be introduced when generatingthe 3D representation. For example, a cut line that accurately excludednon-fetus anatomy (e.g., the umbilical cord or placenta) in the 2Dultrasound frame on which the cut line was defined may inadvertentlyinclude such non-fetus anatomy on another 2D ultrasound frame. This maylead to the non-fetus anatomy being included in the generated 3Drepresentation, and the non-fetus anatomy obscuring the fetus in thegenerated 3D representation. In another example, the cut line thataccurately included all fetus anatomy in the 2D ultrasound frame onwhich the cut line was defined may inadvertently exclude portions of thefetus on another 2D ultrasound frame. This may lead to the resulting 3Drepresentation being incomplete (e.g., omitting a limb or a portion ofthe head of the fetus). Furthermore, movement of the fetus during thescan may also lead to imperfections in the generated 3D representation.In these various examples, the whole process may need to be repeated. Itwould therefore be useful to find a way to reduce the required skilllevel of the operator and to provide improved cut lines on the multiple2D images that are used to generate the 3D representation of the fetus.

The above background information is provided to reveal informationbelieved by the applicant to be of possible relevance to the presentinvention. No admission is necessarily intended, nor should beconstrued, that any of the preceding information constitutes prior artagainst the present invention. The embodiments discussed herein mayaddress and/or ameliorate one or more of the aforementioned drawbacksidentified above. The foregoing examples of the related art andlimitations related thereto are intended to be illustrative and notexclusive. Other limitations of the related art will become apparent tothose of skill in the art upon a reading of the specification and astudy of the drawings herein.

BRIEF DESCRIPTION OF DRAWINGS

The following drawings illustrate embodiments of the invention andshould not be construed as restricting the scope of the invention in anyway.

FIG. 1 is a flowchart of the main steps for generating a 3D fetalrepresentation, according to an embodiment of the present invention.

FIG. 2 is a fetal ultrasound image showing a cut line, according to anembodiment of the present invention.

FIG. 3 is another fetal ultrasound image showing a cut line, accordingto an embodiment of the present invention.

FIG. 4 is a schematic diagram of the training and deployment of an AImodel, according to an embodiment of the present invention.

FIG. 5 is a flowchart for selecting ultrasound frames for the generationof the 3D representation, according to an embodiment of the presentinvention.

FIG. 6 is a flowchart for generating a 4D ultrasound representation,according to an embodiment of the present invention.

FIG. 7A is a flowchart for training an AI model, according to anembodiment of the present invention.

FIG. 7B is another fetal ultrasound image showing a cut line, accordingto an embodiment of the present invention.

FIG. 8 is a flowchart illustrating a fetal ultrasound image with morethan one predicted cut line and resulting generation of a predicted 3Dfetal representation, according to an embodiment of the presentinvention.

FIG. 9 is a flowchart for calculating the age of a fetus, according toan embodiment of the present invention.

FIG. 10 is a schematic diagram of an ultrasound imaging system,according to an embodiment of the present invention.

FIG. 11 is a schematic diagram of a system with multiple ultrasoundscanners, according to an embodiment of the present invention.

DETAILED DESCRIPTION A. Glossary

The term “AI model” means a mathematical or statistical model that maybe generated through artificial intelligence techniques such as machinelearning and/or deep learning. For example, these techniques may involveinputting labeled or classified data into a neural network algorithm fortraining, so as to generate a model that can make predictions ordecisions on new data without being explicitly programmed to do so.Different software tools (e.g., TensorFlow™, PyTorch™, Keras™) may beused to perform machine learning processes.

The term “module” can refer to any component in this invention and toany or all of the features of the invention without limitation. A modulemay be a software, firmware or hardware module, and may be located, forexample, in the ultrasound scanner, a display device or a server.

The term “communications network” can include both a mobile network anddata network without limiting the term's meaning, and includes the useof wireless (e.g. 2G, 3G, 4G, 5G, WiFi™, WiMAX™, Wireless USB (UniversalSerial Bus), Zigbee™, Bluetooth™ and satellite), and/or hard wiredconnections such as local, internet, ADSL (Asymmetrical DigitalSubscriber Line), DSL (Digital Subscriber Line), cable modem, T1, T3,fiber-optic, dial-up modem, television cable, and may includeconnections to flash memory data cards and/or USB memory sticks whereappropriate. A communications network could also mean dedicatedconnections between computing devices and electronic components, such asbuses for intra-chip communications.

The term “operator” (or “user”) may refer to the person that isoperating an ultrasound scanner (e.g., a clinician, medical personnel, asonographer, ultrasound student, ultrasonographer and/or ultrasoundtechnician).

The term “processor” can refer to any electronic circuit or group ofcircuits that perform calculations, and may include, for example, singleor multicore processors, multiple processors, an ASIC (ApplicationSpecific Integrated Circuit), and dedicated circuits implemented, forexample, on a reconfigurable device such as an FPGA (Field ProgrammableGate Array). A processor may perform the steps in the flowcharts andsequence diagrams, whether they are explicitly described as beingexecuted by the processor or whether the execution thereby is implicitdue to the steps being described as performed by the system, a device,code or a module. The processor, if comprised of multiple processors,may be located together or geographically separate from each other. Theterm includes virtual processors and machine instances as in cloudcomputing or local virtualization, which are ultimately grounded inphysical processors.

The term “scan convert”, “scan conversion”, or any of its grammaticalforms refers to the construction of an ultrasound media, such as a stillimage or a video, from lines of ultrasound scan data representing echoesof ultrasound signals. Scan conversion may involve converting beamsand/or vectors of acoustic scan data which are in polar (R-theta)coordinates to cartesian (X-Y) coordinates.

The term “system” when used herein, and not otherwise qualified, refersto a system for generating a 3D fetal representation, the system being asubject of the present invention. In various embodiments, the system mayinclude an ultrasound machine (including a display and one or moretransducers); an ultrasound scanner and a display device; and/or anultrasound scanner, display device and a server.

The term “ultrasound image frame” (or “image frame” or “ultrasoundframe”) refers to a frame of post-scan conversion data that is suitablefor rendering an ultrasound image on a screen or other display device.

B. Exemplary Embodiments

At a high level, the embodiments herein generally allow ultrasoundframes of a fetus to be acquired using an ultrasound scanner, which maybe oriented arbitrarily with respect to the fetus during theacquisition. The ultrasound frames may be processed against anartificial intelligence (AI) model to predict a suitable cut line oneach of the ultrasound frames, where each cut line is positionedexterior to an image of the fetus appearing on the ultrasound frame. Thedifferent cut lines on the ultrasound frames are then used to generate a3D representation of the fetus.

Referring to FIG. 1, the high-level steps of a method for generating a3D fetal representation are shown. In step 10, a number of ultrasoundframes of the fetus may be acquired using an ultrasound scanner(hereinafter “scanner”, “probe”, or “transducer” for brevity). Theultrasound frames may be acquired by fanning a series of a planes (witha frame each containing a sequence of transmitted and receivedultrasound signals), through an angle and capturing a differentultrasound frame at each of a number of different angles. During thescanning, the scanner may be held steady by an operator of the scannerwhile a motor in the head of the scanner tilts the ultrasonic transducerto acquire ultrasound frames at different angles. Additionally oralternatively, other methods of acquiring a series of ultrasound framesmay be employed, such as using a motor to translate (e.g., slide) theultrasonic transducer or rotate it, or manually tilting, translating orrotating the ultrasound scanner. When acquiring the number of ultrasoundframes, it is possible, but not necessary, to initially place thescanner in the midsagittal plane of the fetus. In the examples discussedbelow with respect to FIGS. 2-4, example embodiments are discussed wherethe AI model is trained with (and correspondingly, predict) cut lines ona midsagittal plane view of the fetus.

In various embodiments, the AI model may additionally be trained withcut lines on different views of the fetus. For example, these differentviews may include coronal and/or transverse plane views of the fetus,including views from different angles that combine any of a sagittalplane view, a coronal plane view, or a transverse plane view. In theseembodiments where the AI model is also trained to predict cut lines onthese different views, the scanner may be placed in an arbitraryorientation with respect to the fetus, provided that the scannercaptures at least a portion of the fetus. Additional discussion relatedto the different fetal ultrasound frames that can used for training theAI model is provided below in relation to FIG. 7.

In step 12, a different cut line is predicted on each of the acquiredultrasound frames, where possible. To do this, each ultrasound frame maybe processed against an AI model to predict the respective cut line. Thecut lines may be predicted without the operator of the scanner needingto manually input a region of interest on an ultrasound frame. As eachultrasound frame may have a different image of the fetus, or portion ofthe fetus, then the optimal cut line may vary in position and from frameto frame. The cut lines are predicted by the AI model to be exterior toan image of the fetus in the ultrasound frames.

In step 14, the different cut lines on the ultrasound frames are thenused to eliminate various non-fetal anatomy from each of the ultrasoundframes. The remaining image data can be used as slices that together,generate a 3D representation of the fetus. Additional details about howstep 14 of FIG. 1 can be performed are discussed below.

Referring to FIG. 2, an ultrasound image of a fetus 20 is shown. Thefetus 20 may include the fetal head 22 and fetal abdomen 26. Otheranatomy may also be visible in fetal ultrasound frames. For example,this anatomy may include the uterus, uterine wall, the placenta, theamniotic sac, the umbilical cord, and the amniotic fluid 32. FIG. 2illustrates an example cut line 40 as may initially be used to train anAI model, or as may be predicted by an AI model. In the illustratedexample, the cut line 40 is shown exterior to the fetus 20 and interiorto (e.g., inside) an imaged uterus that surrounds the fetus 20.

The cut line 40 may closely follow the profile of the fetus 20.Depending on the profile of the fetus 20 viewable in a given ultrasoundframe, the nature of the cut line may vary. For example, since theprofile of the fetus 20 viewable in the ultrasound frame is non-smooth,with various protrusions for the head or limbs, the corresponding cutline 40 may correspondingly be uneven and non-smooth so that the cutline 40 contours close to the fetal head 22 (on the right side of theimage), and the fetal abdomen 26 (in the middle of the image).

In other ultrasound frames, if the profile of the fetus 20 appearing onthese other ultrasound frames have fewer or more protrusions and/orindentations, the corresponding cut lines may be smoother or morecomplex than example cut line 40 shown in FIG. 2. Additionally oralternatively, in still other ultrasound frames, the predicted cut linemay be closer or further from the fetus 20 than the predicted cut line40, in one or more different regions of the cut line 40.

Referring to FIG. 3, shown there is another example fetal ultrasoundimage with a different predicted cut line 46. In the ultrasound image ofFIG. 3, profile of the fetus 20 contains only the fetal head 22 and aportion of the fetal abdomen 26. The profile of the fetus 20 does nothave any protrusions for limbs. As a result, the cut line 46 in FIG. 3may be a smoother less jagged line as compared to the cut line 40 ofFIG. 2, while still closely following the profile of the fetus 20. Bothcut lines 40 in FIG. 2 and cut line 46 of FIG. 3 may validly be used forremoval of non-fetal data in the ultrasound frames before the generationof the 3D fetal representation.

In various embodiments, a cut line may generally be a freeform line, sothat the cut line can freely adapt to the various contours of theprofile of a fetus 20 visible in a given ultrasound image frame.However, in some embodiments, a cut line 40 may additionally oralternatively include a Bézier curve, a polynomial curve, a splinecurve, a parametric curve, and/or a more complex curve made up of two ormore of these types of curve. As discussed below, in some embodiments,the cut lines may be part of a shape for masking out the imagednon-fetal anatomy to be removed when generating the 3D fetalrepresentation, or masking out the portion of the ultrasound image to beretained when generating the 3D fetal representation.

Referring back to FIG. 2, the image areas of the ultrasound frame thatare on a distal side 42 of the predicted cut line 40, relative to thefetus 20, may be removed prior to generating the 3D fetalrepresentation. The image areas of the ultrasound frame that are on aproximal side 44 of the predicted cut line 40, relative to the fetus 20,may then be used for the generation of the 3D fetal representation. Byremoving the portions of the ultrasound image on the distal side 42 ofeach ultrasound frame that compose the various slices forming the 3Dfetal representation, portions of non-fetal anatomy will be excludedfrom the resulting 3D fetal representation. Alternately, in someembodiments, the portions of the ultrasound image on the distal side 42of the ultrasound frame may simply be ignored when generating the 3Dfetal representation.

As can be seen in the example of FIG. 2, the AI model may aim to predictthe cut line 40 so that it is exterior to the imaged fetus 20, fetalhead 22, and fetal abdomen 26, and so that it is exterior to (e.g., doesnot lie within) any imaged placenta, uterine wall 24, amniotic sac,umbilical cord, cervix and bladder. The predicted cut line 40 may,however, lie within the imaged amniotic fluid 32.

While amniotic fluid 32 generally has a dark appearance on an ultrasoundimage, simple edge detection techniques simply to trace the amnioticfluid may not accurately identity the profile of the fetus 20. This isbecause the presence of the amniotic fluid 32 may not be clear in allultrasound images. For example, in some ultrasound images, some or allof the fetus 20 may be positioned adjacent to a wall of an amniotic sacsuch that there may not be an identifiable layer of amniotic fluid 32appearing in the ultrasound image. Additionally or alternatively, insome ultrasound images, the umbilical cord may appear as beingcontiguous with the fetus 20 itself, such that an edge detection tracingtechnique may inadvertently include the umbilical cord and thus notidentify a cut line 40 that accurately traces the profile of the fetus20. In the present embodiments, by training an AI model to account forthese different scenarios, the AI model is more likely to predict anaccurate cut line 40 (e.g., a cut line that delineates the fetus 20 fromthe wall of the amniotic sac, and/or delineate the fetus 20 from theumbilical cord). Additional details related to the training of the AImodel are discussed below.

As referenced above with respect to FIG. 3, some ultrasound images maybe of a portion of the fetus rather than the whole fetus. For example,parts of the fetal anatomy that may be imaged in an ultrasound frame mayinclude a face, a head, an ear, a nose, an eye, a neck, a torso, a foot,a leg, a hand, an arm, or a combination of any of these. An AI model maybe trained similarly to predict the profile of the portion of the fetus20 that appears in a given ultrasound image.

Referring to FIG. 4, shown there generally is a schematic diagram fortraining an AI model 56 to predict cut lines on fetal ultrasound imageframes to generate a 3D fetal representation.

The AI model 56 may generally be trained by ultrasound frames that eachhave a labeled cut line that is positioned relative to an imaged fetus20 in the training ultrasound frame. The training ultrasound frames mayinclude ultrasound frames 52 a with cut lines that are tagged asacceptable, and/or ultrasound frames 52 b with cut lines that are taggedas unacceptable. Some of the labeled cut lines for training the AI model56 may be defined using manual input. For example, a cut line on atraining ultrasound frame may be labeled as acceptable if all points onthe cut line are positioned exterior to the imaged fetus, and exteriorto an imaged placenta, uterine wall, amniotic sac, umbilical cord,cervix, and bladder. In another example, a cut line on a trainingultrasound frame may be labeled as acceptable if all points on the cutline are positioned within amniotic fluid imaged in the trainingultrasound frame.

In contrast, a cut line on a training ultrasound frame may be labeled asunacceptable if any point on the cut line is positioned on or interiorto the imaged fetus, or if any point on the cut line is positioned on orinterior to an imaged placenta, uterine wall, amniotic sac, umbilicalcord, cervix, or bladder.

In some embodiments, an optional pre-processing act 50 may be performedon the underlying ultrasound image frames 52 a, 52 b to facilitateimproved performance and/or accuracy when training the machine learning(ML) algorithm. For example, since the desired output of the AI model isa cut line that generally traces the profile boundary of the fetus 20(without needing to consider greyscale details within the variousanatomy), it may be possible to pre-process the ultrasound images 52 a,52 b through a high contrast filter to reduce the granularity ofgreyscale on the ultrasound images 52 a, 52 b.

Additionally or alternatively, since the desired cut line relative tothe ultrasound images 52 a, 52 b should generally remain the sameregardless of the scale of the ultrasound images, it may be possible toreduce scale of the ultrasound images 52 a, 52 b prior to providing theultrasound images 52 a, 52 b to the training step 54. Reducing the scaleof ultrasound images 52 a, 52 b as a preprocessing step may reduce theamount of image data to be processed during the training act 54, andthus may reduce the corresponding computing resources required for thetraining act 54 and/or improve the speed of the training act 54.

Various additional or alternative pre-processing acts may be performedin act 50. For example, these acts may include data normalization toensure that the various ultrasound frames 52 a, 52 b used for traininghave generally the same dimensions and parameters.

Referring still to FIG. 4, the various training frames 52 a, 52 b may,at act 54, be used to train a ML algorithm. For example, the varioustraining ultrasound frames 52 a, 52 b, may be inputted into a deepneural network that can learn how to predict a correct cut line on newultrasound images. For example, the neural network may learn to detectthe presence of the dark border (representative of the amniotic fluid32) between the profile of a fetus 20 and any surrounding tissue so asto position the cutline within that dark border.

The result of the training may be the AI model 56, which represents themathematical weights and/or parameters learned by the deep neuralnetwork to predict an accurate cut line on new fetal ultrasound images.The training act 54 may involve various additional acts (not shown) togenerate a suitable AI model 56. For example, these various deeplearning techniques such as regression, classification, featureextraction, and the like. Any generated AI models may be iterativelytested to ensure they are not overfitted and sufficiently generalizedfor identifying cut lines on new fetal ultrasound images. In variousembodiments, the machine learning may be supervised or unsupervised.

For example, in some embodiments, once the training images are labelled,a deep neural network may use them as inputs and the associated expertcut-line as desired may be outputted to determine value sets of neuralnetwork parameters defining the neural networks.

As noted above, in some embodiments, the cut lines may be part of ashape that masks out the imaged non-fetal anatomy or the imaged fetus.For example, referring briefly simultaneously to FIG. 7B, the cut line40 b shown there is not simply a line, but rather as a closed shape thatencompasses areas of the non-fetal anatomy above the cut line. Referringback to FIG. 4, in some embodiments, the neural network may beconfigured to receive one or more ultrasound images as input and to havea softmax layer as an output layer containing outputs which representwhether pixels of the corresponding input image fall within the areaabove the cut-line or not.

The training images file may include an image identifier field forstoring a unique identifier for identifying an image included in thefile, a segmentation mask field for storing an identifier for specifyingthe to-be-trimmed area, and an image data field for storing informationrepresenting the image.

In some embodiments, using a cross-validation method on the trainingprocess would optimize neural network hyper-parameters to try to ensurethat the neural network can sufficiently learn the distribution of allpossible fetus image types without overfitting to the training data. Insome embodiments, after finalizing the neural network architecture, theneural network may be trained on all of the data available in thetraining image files.

In various embodiments, batch training may be used and each batch mayconsist of multiple images, thirty-two for example, wherein each exampleimage may be gray-scale, 256*256 pixels, without any preprocessingapplied to it.

In some embodiments, the deep neural network parameters may be optimizedusing the adam optimizer with hyper-parameters as suggested by Kingma,D. P., Ba, J. L.: Adam: a Method for Stochastic Optimization,International Conference on Learning Representations 2015 pp. 1-15(2015), the entire contents of which are incorporated herewith. Theweight of the convolutional layers may be initialized randomly from azero-mean Gaussian distribution. In some embodiments, the Keras™ deeplearning library with TensorFlow™ backend may be used to train and testthe models.

In some embodiments, during training, many steps may be taken tostabilize learning and prevent the model from over-fitting. Using theregularization method, e.g., adding a penalty term to the loss function,has made it possible to prevent the coefficients or weights from gettingtoo large. Another method to tackle the over-fitting problem is dropout.Dropout layers limit the co-adaptation of the feature extracting blocksby removing some random units from the neurons in the previous layer ofthe neural network based on the probability parameter of the dropoutlayer. Moreover, this approach forces the neurons to follow overallbehaviour. This implies that removing the units would result in a changein the neural network architecture in each training step. In otherwords, a dropout layer performs similar to adding random noise to hiddenlayers of the model. A dropout layer with the dropout probability of 0.5may be used after the pooling layers.

Data augmentation is another approach to prevent over-fitting and addmore transitional invariance to the model. Therefore, in someembodiments, the training images may be augmented on-the-fly whiletraining. In every mini-batch, each sample may be translatedhorizontally and vertically, rotated and/or zoomed, for example.

Referring still to FIG. 4, after training has been completed, the setsof parameters stored in the storage memory may represent a trainedneural network for masking out the imaged portions of the fetus.

In order to assess the performance of the model, the stored modelparameter values can be retrieved any time to perform image assessmentthrough applying an image to the neural networks represented thereby.

In some embodiments, the deep neural network may include various layerssuch as convolutional layers, max-pooling layers, and fully connectedlayers. In some embodiments, the final layers may include a softmaxlayer as an output layer having outputs which eventually woulddemonstrate respective determinations that an input set of pixels fallwithin a particular area above or below the cut-line. Accordingly, insome embodiments, the neural network may take at least one image as aninput and output a binary mask indicating which pixels belong to thearea above the cut-line (e.g., the AI model classifies which area eachpixel belongs to).

To increase the robustness of the AI model 56, in some embodiments, abroad set of training data may be used at act 54. For example,ultrasound images at different gestational fetus ages can be included inthe ultrasound images 52 a, 52 b used for training. For example, thetraining data may include ultrasound images for early obstetrics (OB)(up to 8 weeks), mid OB (between 8 and 26 weeks) and late OB (after 26weeks). Cut lines may then be identified on the ultrasound images ofthese various fetus ages while training, so that the AI model 56 canlearn to generate the cut line regardless of the age of the fetus.

Additionally or alternatively, the training act 54 in FIG. 4 may berepeated with different datasets for the different fetus ages togenerate corresponding different AI models 56 that each generate cutlines for fetus age-specific ultrasound images. In this scenario, theappropriate AI model can be selected to be applied based on the age ofthe fetus. The age of the fetus can be determined in various ways: forexample, based on patient examination information, OB measurements takenthat correlate to age, and/or another AI model trained to identify fetusage of an imaged fetus.

In these example embodiments where the training datasets include fetusesof different ages, the resulting AI model(s) 56 may be more robust. Thisis because later-stage fetuses may have grown to a stage where it is noteasy to delineate between the profile of the fetus 20 and thesurrounding tissue (e.g., the usually dark appearance of the amnioticfluid 32 may not be as apparent on the ultrasound images). By having thedataset include these scenarios, the AI model 56 (if only a single AImodel 56 is generated) may learn to recognize these situations so as toproduce accurate cut lines when scanning fetuses that are in laterpregnancy stages. Or, if the different fetus-age datasets are used togenerate different corresponding AI models 56, then the potentially morechallenging cut lines to predict for ultrasound images of late OBfetuses can be avoided when training the earlier-stage fetus images.This may make the AI models 56 for the earlier-stage fetuses easier totrain, so that they are more likely to converge on an accurate AI model56 for ultrasound images of early OB and mid OB fetuses. The early OBand mid OB AI models my then be more confidently used with early OB andmid OB stage fetuses.

Referring still to FIG. 4, once a satisfactory AI model 56 is generated,the AI model 56 may be deployed for execution on a neural network 58 topredict cut lines on new fetal ultrasound images 60. Notably, the neuralnetwork 58 is shown in FIG. 4 for illustration as a convolution neuralnetwork—with various nodes in the input layer, hidden layers, and outputlayers. However, in various embodiments, different arrangements of theneural network 58 may be possible.

In various embodiments, prior to being processed for prediction of cutlines thereon, the new ultrasound images 60 may optionally bepre-processed. This is shown in FIG. 4 with the pre-processing act 50 indotted outline. In some embodiments, these pre-processing acts 50 may beanalogous to the pre-processing acts 50 performed on the trainingultrasound frames 52 a, 52 b (e.g., processing through a high contrastfilter and/or scaling), to facilitate improve accuracy in generating apredicted cut line.

In various embodiments, the new fetal ultrasound images 60 may be liveimages acquired by an ultrasound imaging system (e.g., the systemdiscussed with respect to FIG. 10 below). For example, the AI model 56may be deployed for execution on the scanner 131 and/or the displaydevice 150 discussed in more detail below. Additionally oralternatively, the AI model 56 may be executed on stored images 60 thatwere previously acquired (e.g., as may be stored on a PicturingArchiving and Communication System (PACS)). When executed in thismanner, the AI model 56 may allow the neural network 58 to predict cutlines on the new ultrasound frames 60, resulting in a series ofultrasound frames 62 with predicted cut lines defined thereon. Theultrasound frames 62 with predicted cut lines may then be used for thegeneration of a 3D representation 64 of the fetus. As shown in FIG. 4for illustrative purposes, the 3D representation 64 is a 3D surface ofthe face of the fetus 20. However, in various embodiments, additionalanatomical features of the fetus 20 may also be shown in the 3Drepresentation 64.

In some embodiments, the ultrasound frames 62 with predicted cut linesmay optionally each be labeled as either acceptable or unacceptable, andthese labeled ultrasound frames may themselves be used for trainingand/or reinforcing the AI model 56. For example, the frames shown inFIGS. 2 and 3 may be used for training and/or reinforcing the AI model56. This is shown in FIG. 4 with a dotted line from the ultrasoundframes 62 with the predicted cut lines being provided to the trainingact 54.

Referring to FIG. 5, a method for selecting ultrasound frames for thegeneration of a 3D fetal representation is shown. The first step 12 maycorrespond to the second step in FIG. 1, in which a different cut lineis predicted on each of the acquired fetal ultrasound frames, wherepossible. As part of the process for generating the 3D fetalrepresentation, after a cut line has been predicted, the AI model 56 maydetermine a confidence level for the prediction of each cut line, instep 80. For example, the confidence level may be rated high (e.g. above70%) if the AI model 56 deems that there is a generally low level ofuncertainty in the possible positioning of the predicted cut line. Incontrast, the confidence level may be rated low (e.g. 50% or below) ifthe AI model 56 deems that there is a relatively higher level ofuncertainty in the possible positioning of the predicted cut line.Higher levels of uncertainty may arise, for example, for ultrasoundframes in which two adjacent body parts have the same or a similar imagedensity on the ultrasound frame.

In step 82, the ultrasound frames for which the confidence level isabove a threshold may then be used to generate the 3D fetalrepresentation, with the other ultrasound frames for which theconfidence level is below the threshold being discarded or ignored.

For example, the process may include removing, from each of theultrasound frames for which the confidence level is above the threshold,ultrasound data on a distal side of the cut line relative to the imageof the fetus appearing on the ultrasound frame. The remaining ultrasounddata in each of the ultrasound frames from which ultrasound data hasbeen removed may then be used to generate the 3D fetal representation.

By using only the ultrasound frames with cut lines for which theconfidence level is above the threshold, the calculations required forgenerating the 3D fetal representation may more likely be accurate. Forexample, the resultant 3D fetal representation may less likely includenon-fetal anatomy and/or cut off portions of the fetal anatomy than ifthe full set of ultrasound frames were used. Since frames for which thepredicted cut line is below the confidence threshold may be dropped, itis possible that there may be gaps in the various slices that form the3D fetal representation. To reduce the potential appearance ofjaggedness, interpolation and/or smoothing algorithms may be used whengenerating the 3D fetal representation.

In some embodiments, instead of discarding or ignoring the ultrasoundframes for which the confidence is below the threshold, the predictedcut line for a nearby frame may be used for these ultrasound frames. Forexample, it may be the case that the confidence level for the cut linepredicted for an adjacent frame (e.g., a frame acquired immediatelybefore or after) or a neighboring frame (e.g., a frame acquired with acertain number of frames) is above the acceptance threshold, then thecut line predicted on that adjacent or neighboring frame can be used onthe ultrasound frame for which the predicted cut line has not met theconfidence threshold. Such a configuration may allow more ultrasoundframes to be included in the slices that form the 3D fetalrepresentation and reduce the possibility of the generated 3D fetalrepresentation appearing jagged (as may be the case if too manyultrasound frames are discarded or ignored when generating the 3D fetalrepresentation).

Referring to FIG. 6, a method is shown for generating a 4D fetalultrasound representation, which may be a 3D representation that variesin real time as the fetus moves. The first three steps 10, 12, 14 may bethe same as those in FIG. 1, in which a number of ultrasound frames areacquired, cut lines are predicted and a 3D fetal representation isgenerated. In step 90, the generated 3D fetal representation may then beprojected into a real time image that is displayed. The process thenreverts to step 10 to repeat the cycle. Every time the cycle isrepeated, the real time image may be updated with the most recent 3Dfetal representation that is generated.

As noted above, in some embodiments, a motor in the head of the scannermay tilt the ultrasonic transducer to acquire ultrasound frames atdifferent angles. In various embodiments, the motor may perform a sweepof the possible angles of acquisition at a frame rate of 15-40 framesper second (fps). These frames can be included in the slices forming the3D fetal representations, so that resultant 4D fetal representations canhave a volume rate of 1-3 volumes per second (vps). In variousembodiments, the 4D fetal frame rate can vary depending on the number ofslices included in each 3D fetal representation. For example, if fewerindividual ultrasound frames are included as slices when generating the3D fetal representation, the 4D volume rate may be improved (though eachindividual frame may not have as much resolution). Conversely, ifincreased resolution in the 3D fetal representation is desired, a highernumber of individual ultrasound frames may be included as slices whengenerating each 3D fetal representation, and the corresponding 4D volumerate may be decreased.

Referring to FIG. 7A, a flowchart for training the AI model 50 is shown,according to an example embodiment. For example, the acts of FIG. 7A maybe performed as part of steps to train the ML algorithm (e.g., as partof act 54 of FIG. 4), where ultrasound frames with manually-defined cutlines 52 a, 52 b and/or ultrasound frames with AI-predicted cut lines 62can be provided to train the AI model.

In FIG. 7A, ultrasound frames with AI-predicted cut lines may beprovided in step 100 and ultrasound frames with manually defined cutlines may be provided in step 102. In step 104, for each ultrasoundframe provided, a determination may be made as to whether the cut lineis exterior to the imaged fetus 20 (e.g., as shown in FIG. 2). If thecut line is exterior to the imaged fetus 20, then, in step 106, it maybe determined whether the cut line is exterior to other body tissue,e.g., body tissue of the mother that is imaged on the ultrasound frame.If the cut line is exterior to the other body tissue, then, in step 110,the ultrasound image with the cut line may be labeled as acceptable. Theultrasound image with the acceptable cut line may then be used to trainthe AI model in step 54 of FIG. 4.

Returning to step 104, if the cut line is not exterior to the imagedfetus, then the cut line is not acceptable, and the correspondingultrasound frame with the cut line may be labeled as unacceptable instep 114. The ultrasound image with the unacceptable cut line may thenbe used to train the AI model at step 54.

Returning to step 106, if the cut line is not exterior to other imagedbody tissue, then the cut line is not acceptable, and the correspondingultrasound frame with the cut line may be labeled as unacceptable instep 114. The ultrasound image with the unacceptable cut line may thenbe used to train the AI model 50 in step 54 of FIG. 4.

Referring still to FIG. 7A, an additional or alternative possibility toanalyze the ultrasound frames with AI-predicted cut lines provided instep 100 and ultrasound frames with manually defined cut lines in step102 is shown in step 120. In step 120, it is determined whether the cutline lies within the imaged amniotic fluid 32 (e.g., as shown in FIG.2). If the cut line lies within the amniotic fluid 32, then theultrasound frame with the cut line may be labeled as acceptable in step110. The ultrasound image with the acceptable cut line may then be usedto train the AI model 50 in step 54.

If, in step 120, the cut line does not lie within the amniotic fluid,then the ultrasound frame with the cut line may be labeled asunacceptable in step 114. The ultrasound image with the unacceptable cutline may then be used to train the AI model 50 in step 54 of FIG. 4.

It can therefore be seen that there are a variety of different ways inwhich an ultrasound frame may be analyzed. Other ways may be possible inother embodiments. For example, while FIG. 7A shows step 120 being analternative path to steps 104 and 106 for determining cut lineacceptability, in various embodiments, the step 120 may be may performedin addition to the determinations made at steps 104 and 106.

As discussed above in relation to FIG. 1, to increase the robustness ofthe AI model, the ultrasound frames used for training may includevarious views of a prenatal fetus. In various embodiments, theacquisition protocol for acquiring the training ultrasound frames mayfirst specify that the ultrasound operator is to acquire a frontalmidsagittal view of the fetus 20 (e.g., as is shown in FIG. 2). Oncethis view has been acquired, the acquisition protocol may then indicatethat additional frames are to be acquired as the probe is rotated at anumber of increments until a transverse view of the fetus is acquired,and then continuing with additional rotational increments until theprobe has returned to a frontal midsagittal view of the fetus. In anexample, the increments may be 45 degrees, 90 degrees (e.g., atransverse view), then 135 degrees, then 180 degrees; so that a total of4 different planes are acquired. The ultrasound frames from thesevarious views can then be used to identify cut lines, for training theAI model as discussed herein.

In some embodiments, this rotational protocol may also be performed notjust from a frontal midsagittal view of the fetus, but also from one ormore of: a rear midsagittal view of the fetus (e.g., when the probe headis facing the backside of the fetus 20), a left coronal view of thefetus 20 (e.g., when the probe head is facing the left side of the fetus20), a right coronal view of the fetus 20 (when the probe head is facingthe right side of the fetus 20), a superior coronal view of the fetus 20(when the probe head is facing the head of the fetus 20), and aposterior coronal view of the fetus 20 (when the probe head is facingthe feet of the fetus 20). Cut lines can be identified on these variousviews of the fetus for training the AI model 56, so that the AI model 56may be able to predict cut lines when corresponding views of the fetusare obtained on newly acquired ultrasound images 60.

Unskilled or novice ultrasound operators may not have developed theskill to obtain a traditional midsagittal view of the fetus 20 thatprovides an accurate profile of the fetus 20. Thus, training the AImodel 56 with such off-angle ultrasound images may increase therobustness of the AI model 56, so as to be operational when fetalultrasound images are acquired by unskilled or novice operators.

For example, referring to FIG. 7B, shown there is another example fetalultrasound image with a cut line, according to an embodiment of thepresent invention. In FIG. 7B, the image acquired of the fetus 20 isrotationally off angle from a midsagittal view that would generallyprovide a better view of the profile of the fetus 20. An example cutline 40 b is shown that delineates between anatomy of the fetus 20 belowthe cut line 40 b and non-fetal anatomy above the cut line 40 b that isnot to be included when generating the 3D fetal representation. Similarto the ultrasound image shown in FIG. 2, the image areas of theultrasound frame that are on a distal side 42 of the cut line 40 b,relative to the fetus 20, may be removed prior to generating the 3Dfetal representation. The image areas of the ultrasound frame that areon a proximal side 44 of the cut line 40 b, relative to the fetus 20,may then be used for the generation of the 3D fetal representation.

In various embodiments, these different views from the acquisitionprotocol may all be obtained for fetuses at the different gestationalages discussed above. As noted, the ability to identify a cut line maybe easier for early OB and mid OB fetuses, than for late OB fetuses.Thus, obtaining these various views for fetuses of the variousgestational ages discussed above can increase the performance of thetrained AI model 56, so that predicted cut lines will more likely beaccurate. As noted above, in FIG. 7B, the cut line 40 b is shown as partof a shape that masks out the non-fetal anatomy to be removed prior togenerating the 3D fetal representation.

By training the AI model 56 to predict cut lines on a wide variety ofviews of the fetus, the present embodiments can reduce the time andskill needed by an operator to obtain a 3D fetal representation. Intraditional methods of generating a 3D fetal representation, anultrasound operator typically first identifies a view of the fetus 20 asclose to the midsagittal view as possible (which takes time to positionthe ultrasound probe). Once this view is found, the operator mayposition a Region Of Interest (ROI) for placement of the fixed cut linefor generating the 3D fetal representation. Since it is generallydesired to get a 3D view of the face of the fetus 20, and since theposition of the probe is often not directly over the center of the frontview of the fetus 20, the operator typically rotates the 3D fetalrepresentation after it is generated so that the complete frontal viewof the fetus can be viewable. These various steps in a traditionalmethod of generating a 3D fetal representation are lengthy andcumbersome.

In contrast, in the present embodiments, since the AI model 56 istrained to detect cut lines on a wide variety of views of the fetus, theAI model 56 can immediately be applied when at least a portion of thefetus is scanned by the ultrasound probe—without the operator having tofirst identify the frontal midsagittal view of the fetus. In thismanner, when the probe being placed in an arbitrary orientation withrespect to the fetus 20, a 3D fetal representation of that portion ofthe anatomy can generally be generated and visualized—regardless ofwhether it is a frontal, rear, side, superior, or posterior view of thefetus 20, or any blend of these views.

Moreover, since the AI model 56 can work to immediately apply a cut linein real-time, it is possible to generate a 3D representation of thefetus as the probe moves over the surface of the skin. In this manner,the resultant generated 3D fetus representation may providevisualization that works similar to what may be expected if a flashlightwas able to “see” through the surface of the skin. E.g., the 3Drepresentation of the fetus may be in line with the center of the probehead, so as to give an operator a live 3D representation of the fetusthat is in the direct field of view of the center of the ultrasoundprobe.

Such a “flashlight” mode of operation may be different from thetraditional methods of generating a 3D representation of the fetus notedabove, where during the various steps required to generate the 3D fetalrepresentation, the fetus may move or the generated 3D fetalrepresentation may need to be rotated to permit viewing of the face,such that the generated 3D visualization may not align with the centerof the probe. Having the generated 3D visualization correspond with thecenter of the probe in real-time as the probe is being moved may bedesirable in various situations. For example, such a mode of operationmay be desirable for the operator to more accurately identify theposition of the face in a real-time during delivery of a baby, since thefetus should generally be facing up during delivery and not facing down.

In various embodiments, when visualizing the 3D fetal representation,different ultrasound modes to highlight different anatomy may be used.For example, in some embodiments, it may be possible to image in askeleton mode that highlights the bone structure of the fetus 20. Thiswould allow the bone structure to be visible in the generated 3D fetalrepresentation. Such a skeleton mode may be different from a regularskin realistic mode for visualizing the 3D fetal representation.

In the embodiments discussed above, the 3D fetal representation isgenerally a 3D surface that follows the different cut lines predicted bythe AI model 56 for the various slices formed from the underlying 2Dultrasound images. However, in some embodiments, it may be possible togenerate a full 3D volumetric representation of the fetus 20 thatprovides not just a 3D surface showing the side of the fetus proximateto the transducer head. For example, this may be performed if multiplecut lines are included when training the AI model 56, so that the AImodel 56 can predict multiple cut lines on new ultrasound images 60.

Referring to FIG. 8 shown there generally is a flowchart of a fetalultrasound image with more than one predicted cut line, and resultinggeneration of a predicted 3D fetal volume. The fetal ultrasound imageshown in FIG. 8 is the same as that shown in FIG. 2, except in additionto the cut line 40 shown in FIG. 2, the ultrasound image is shown with asecond cut line 40 a. This second cut line 40 a may delineate the fetus20 from the non-fetal anatomy on the distal side of the fetus 20relative to the probe head. In various embodiments, the training data(e.g., as discussed above with respect to FIG. 4) may include insertingsuch an additional cut line 40 a, so that the two cut lines together 40,40 a, delineate the fetus 20 from non-fetal anatomy on both theproximate and distal sides of the fetus 20 relative to the probe head.The AI model 56 may correspondingly be trained to identify the multiplecut lines 40, 40 a on new fetal ultrasound images.

When multiple cut lines 40, 40 a are identified by the AI model 56 oneach ultrasound frame, this may allow only the imaged fetus 20 betweenthe cut lines 40, 40 a to be retained when generating the 3D fetalrepresentation. When this fetal ultrasound information is used as slicesacross various ultrasound image frames, the generated 3D fetalrepresentation may be a 3D fetal volume 64 a that shows the 3D contoursof the fetus 20 both on the proximate side of the fetus 20 relative tothe probe head, and also on the distal side of the fetus 20 relative tothe probe head. In the example of FIG. 8, the proximate side of thefetus 20 relative to the probe head is the front side of the fetus 20,but in various embodiments, the proximate side of the fetus 20 relativeto the probe head may be the back side of the fetus 20 or any otherperspective view of the fetus. As discussed above, the trainingultrasound image frames may include various views of the fetus 20, andmultiple cut lines may be defined on these various views (e.g., fortraining or prediction by the AI model 56).

Having generated the 3D fetal volume, in some embodiments, the 3D fetalvolume may be manipulated during visualization (e.g., rotated alongvarious axis) so that different aspects of the fetus 20 can be seen on adisplay device. In this manner, the visualization of the 3D fetal volumemay be different from the “flashlight” mode discussed above, because theview of the 3D fetal volume may not necessarily line up with the centerof the probe.

Referring to FIG. 9, a flowchart is shown for calculating the age of afetus. From the 3D fetal ultrasound volume, a physical dimension of thefetus may be determined, in step 124. For example, these dimensions maycorrespond to standard OB measurements that may otherwise requirespecific 2D OB ultrasound views to be acquired by an operator. Invarious embodiments, the physical dimensions that is determined may bethe crown rump length (CRL), biparietal diameter (BPD), headcircumference (HC), abdominal circumference (AC), femur length (FL),cephalic index (CI), humerus length (HL), binocular distance (BOD),and/or intraocular diameter (IOD). More than one physical dimension maybe determined from the 3D fetal ultrasound volume. In this manner, thestandard OB exam may be simplified as the operator may not need to spendtime to acquire the various standard views to obtain these variousmeasurements. Instead, these measurements may be obtained by simplyacquiring frames that allow the generation of a 3D fetal volume asdiscussed herein.

In some embodiments, after a physical dimension of the fetus has beendetermined, an age of the fetus may be calculated using the physicaldimension (step 126) since in general, there is a correspondence betweenthe age of a fetus and its physical dimensions. Determination of the ageof the fetus may therefore be performed by the present system withouthuman input to measure the physical dimension.

Referring to FIG. 10, an exemplary system 130 is shown for generating a3D fetal representation. The system 130 includes an ultrasound scanner131 with a processor 132, which is connected to a non-transitorycomputer readable memory 134 storing computer readable instructions 136,which, when executed by the processor 132, may cause the scanner 131 toprovide one or more of the functions of the system 130. Such functionsmay be, for example, the acquisition of ultrasound data, the processingof ultrasound data, the scan conversion of ultrasound data, thetransmission of ultrasound data or ultrasound frames to a display device150, the detection of operator inputs to the ultrasound scanner 131,and/or the switching of the settings of the ultrasound scanner 131.

Also stored in the computer readable memory 134 may be computer readabledata 138, which may be used by the processor 132 in conjunction with thecomputer readable instructions 136 to provide the functions of thesystem 130. Computer readable data 138 may include, for example,configuration settings for the scanner 131, such as presets thatinstruct the processor 132 how to collect and process the ultrasounddata for a given body part (e.g., skeleton mode), and how to acquire aseries of ultrasound frames for the generation of a 3D fetalrepresentation.

The scanner 131 may include an ultrasonic transducer 142 and a motor 143that tilts the ultrasonic transducer 142 relative to the body of thescanner 131 in order to acquire ultrasound frames at a range of angleswhile the scanner 131 is held steady by an operator.

The scanner 131 may include a communications module 140 connected to theprocessor 132. In the illustrated example, the communications module 140may wirelessly transmit signals to and receive signals from the displaydevice 150 along wireless communication link 144. The protocol used forcommunications between the scanner 131 and the display device 150 may beWiFi™ or Bluetooth™, for example, or any other suitable two-way radiocommunications protocol. In some embodiments, the scanner 131 mayoperate as a WiFi™ hotspot, for example. Communication link 144 may useany suitable wireless communications network connection. In someembodiments, the communication link between the scanner 131 and thedisplay device 150 may be wired. For example, the scanner 131 may beattached to a cord that may be pluggable into a physical port of thedisplay device 150.

In various embodiments, the display device 150 may be, for example, alaptop computer, a tablet computer, a desktop computer, a smart phone, asmart watch, spectacles with a built-in display, a television, a bespokedisplay or any other display device that is capable of beingcommunicably connected to the scanner 131. The display device 150 mayhost a screen 152 and may include a processor 154, which may beconnected to a non-transitory computer readable memory 156 storingcomputer readable instructions 158, which, when executed by theprocessor 154, cause the display device 150 to provide one or more ofthe functions of the system 130. Such functions may be, for example, thereceiving of ultrasound data that may or may not be pre-processed; scanconversion of received ultrasound data into an ultrasound image;processing of ultrasound data in image data frames; the display of auser interface; the control of the scanner 131; the display of anultrasound image on the screen 152; the prediction of cut lines onultrasound frames; the generation of 3D fetal representations usingultrasound frames with cut lines; the generation of 4D fetalrepresentations; and/or the storage, application, reinforcing and/ortraining of an AI model 56 that predicts cut lines on ultrasound frames.

Also stored in the computer readable memory 156 may be computer readabledata 160, which may be used by the processor 154 in conjunction with thecomputer readable instructions 158 to provide the functions of thesystem 130. Computer readable data 160 may include, for example,settings for the scanner 131, such as presets for acquiring ultrasounddata; settings for a user interface displayed on the screen 152; and/ordata for one or more AI models 56 for predicting cut lines in fetalultrasound frames. Settings may also include any other data that isspecific to the way that the scanner 131 operates or that the displaydevice 150 operates.

It can therefore be understood that the computer readable instructionsand data used for controlling the system 130 may be located either inthe computer readable memory 134 of the scanner 131, the computerreadable memory 156 of the display device 150, and/or both the computerreadable memories 134, 156.

The display device 150 may also include a communications module 162connected to the processor 154 for facilitating communication with thescanner 131. In the illustrated example, the communications module 162wirelessly transmits signals to and receives signals from the scanner131 on wireless communication link 144. However, as noted, in someembodiments, the connection between scanner 131 and display device 150may be wired.

Referring to FIG. 11, a system 200 is shown in which there are multiplesimilar or different scanners 131, 202, 204 connected to theircorresponding display devices 150, 206, 208 and either connecteddirectly, or indirectly via the display devices, to a communicationsnetwork 210, such as the internet. The scanners 131, 202, 204 may beconnected onwards via the communications network 210 to a server 220.

The server 220 may include a processor 222, which may be connected to anon-transitory computer readable memory 224 storing computer readableinstructions 226, which, when executed by the processor 222, cause theserver 220 to provide one or more of the functions of the system 200.Such functions may be, for example, the receiving of fetal ultrasoundframes, the processing of ultrasound data in ultrasound frames, thecontrol of the scanners 131, 202, 204, the prediction of cut lines onultrasound frames, and/or machine learning activities related to one ormore AI models 56 (as discussed above in relation to FIG. 4). Suchmachine learning activities may include the training and/or reinforcingof one or more AI models 56 for predicting cut lines in fetal ultrasoundframes.

Also stored in the computer readable memory 224 may be computer readabledata 228, which may be used by the processor 222 in conjunction with thecomputer readable instructions 226 to provide the functions of thesystem 200. Computer readable data 228 may include, for example,settings for the scanners 131, 202, 204 such as preset parameters foracquiring ultrasound data, settings for user interfaces displayed on thedisplay devices 150, 206, 208, and data for one or more AI models 56.For example, the AI model 50 may be used to predict cut lines on fetalultrasound frames, as discussed above. Settings may also include anyother data that is specific to the way that the scanners 131, 202, 204operate or that the display devices 150, 206, 208 operate.

It can therefore be understood that the computer readable instructionsand data used for controlling the system 200 may be located either inthe computer readable memory of the scanners 131, 202, 204, the computerreadable memory of the display devices 150, 206, 208, the computerreadable memory 224 of the server 220, or any combination of theforegoing locations.

As noted above, even though the scanners 131, 202, 204 may be different,each fetal ultrasound frame acquired may be used by the AI model 56 fortraining. Likewise, the fetal ultrasound frames acquired by theindividual scanners 131, 202, 204 may all be processed against the AImodel 56 for prediction of the cut lines and/or for reinforcement of theAI model 56.

In some embodiments, the AI models 56 present in the display devices150, 206, 208 may be updated from time to time from an AI model 56present in the server 220, where the AI model present in the server iscontinually trained using ultrasound frames with cut lines acquired bymultiple scanners 131, 202, 204.

Embodiments of the invention may be implemented using specificallydesigned hardware, configurable hardware, programmable data processorsconfigured by the provision of software (which may optionally include‘firmware’) capable of executing on the data processors, special purposecomputers or data processors that are specifically programmed,configured, or constructed to perform one or more steps in a method asexplained in detail herein and/or combinations of two or more of these.Examples of specifically designed hardware are: logic circuits,application-specific integrated circuits (“ASICs”), large scaleintegrated circuits (“LSIs”), very large scale integrated circuits(“VLSIs”) and the like. Examples of configurable hardware are: one ormore programmable logic devices such as programmable array logic(“PALs”), programmable logic arrays (“PLAs”) and field programmable gatearrays (“FPGAs”). Examples of programmable data processors are:microprocessors, digital signal processors (“DSPs”), embeddedprocessors, graphics processors, math co-processors, general purposecomputers, server computers, cloud computers, main computers, computerworkstations, and the like. For example, one or more data processors ina control circuit for a device may implement methods as described hereinby executing software instructions in a program memory accessible to theprocessors.

While processes or blocks are presented in a given order, alternativeexamples may perform routines having steps, or employ systems havingblocks, in a different order, and some processes or blocks may bedeleted, moved, added, subdivided, combined, and/or modified to providealternative or subcombinations. Each of these processes or blocks may beimplemented in a variety of different ways. Also, while processes orblocks are at times shown as being performed in series, these processesor blocks may instead be performed in parallel, or may be performed atdifferent times.

The embodiments may also be provided in the form of a program product.The program product may include any non-transitory medium which carriesa set of computer-readable instructions which, when executed by a dataprocessor, cause the data processor to execute a method of theinvention. Program products according to the invention may be in any ofa wide variety of forms. The program product may include, for example,non-transitory media such as magnetic data storage media includingfloppy diskettes, hard disk drives, optical data storage media includingCD ROMs, DVDs, electronic data storage media including ROMs, flash RAM,EPROMs, hardwired or preprogrammed chips (e.g., EEPROM semiconductorchips), nanotechnology memory, or the like. The computer-readablesignals on the program product may optionally be compressed orencrypted.

Where a component (e.g. software, processor, assembly, device, circuit,etc.) is referred to above, unless otherwise indicated, reference tothat component (including a reference to a “means”) should beinterpreted as including as equivalents of that component any componentwhich performs the function of the described component (i.e., that isfunctionally equivalent), including components which are notstructurally equivalent to the disclosed structure which performs thefunction in the illustrated exemplary embodiments of the invention.

Specific examples of systems, methods and apparatus have been describedherein for purposes of illustration. These are only examples. Thetechnology provided herein can be applied to systems other than theexample systems described above. Many alterations, modifications,additions, omissions and permutations are possible within the practiceof this invention. This invention includes variations on describedembodiments that would be apparent to the skilled addressee, includingvariations obtained by: replacing features, elements and/or acts withequivalent features, elements and/or acts; mixing and matching offeatures, elements and/or acts from different embodiments; combiningfeatures, elements and/or acts from embodiments as described herein withfeatures, elements and/or acts of other technology; and/or omittingcombining features, elements and/or acts from described embodiments. Insome embodiments, the components of the systems and apparatuses may beintegrated or separated. Moreover, the operations of the systems andapparatuses disclosed herein may be performed by more, fewer, or othercomponents and the methods described may include more, fewer, or othersteps. In other instances, well known elements have not been shown ordescribed in detail and repetitions of steps and features have beenomitted to avoid unnecessarily obscuring the invention. Screen shots mayshow more or less than the examples given herein. Accordingly, thespecification is to be regarded in an illustrative, rather than arestrictive, sense.

It is therefore intended that the appended claims and claims hereafterintroduced are interpreted to include all such modifications,permutations, additions, omissions and subcombinations as may reasonablybe inferred. The scope of the claims should not be limited by theembodiments set forth in the examples but should be given the broadestinterpretation consistent with the description as a whole.

C. Interpretation of Terms

Unless the context clearly requires otherwise, throughout thedescription and the claims, the following applies:

In general, unless otherwise indicated, singular elements may be in theplural and vice versa with no loss of generality. The use of themasculine can refer to masculine, feminine or both.

The terms “comprise”, “comprising” and the like are to be construed inan inclusive sense, as opposed to an exclusive or exhaustive sense, thatis to say, in the sense of “including, but not limited to”.

The terms “connected”, “coupled”, or any variant thereof, means anyconnection or coupling, either direct or indirect, between two or moreelements; the coupling or connection between the elements can bephysical, logical, or a combination thereof.

The words “herein,” “above,” “below” and words of similar import, whenused in this application, refer to this application as a whole and notto any particular portions of this application.

The word “or” in reference to a list of two or more items covers all ofthe following interpretations of the word: any of the items in the list,all of the items in the list and any combination of the items in thelist.

Words that indicate directions such as “vertical”, “transverse”,“horizontal”, “upward”, “downward”, “forward”, “backward”, “inward”,“outward”, “vertical”, “transverse”, “left”, “right”, “front”, “back”,“top”, “bottom”, “below”, “above”, “under”, and the like, used in thisdescription and any accompanying claims (where present) depend on thespecific orientation of the examples described and illustrated. Thesubject matter described herein may assume various alternativeorientations. Accordingly, these directional terms are not strictlydefined and should not be interpreted narrowly.

To aid the Patent Office and any readers of any patent issued on thisapplication in interpreting the claims appended hereto, applicant wishesto note that they do not intend any of the appended claims or claimelements to invoke 35 U.S.C. 112(f) unless the words “means for” or“step for” are explicitly used in the particular claim.

D. Claim Support

Disclosed herein is a method for defining cut lines to generate a threedimensional (“3D”) representation of a fetus, the method comprising:acquiring a plurality of ultrasound frames of the fetus using anultrasound scanner in an arbitrary orientation with respect to thefetus; and processing, by a processor, each ultrasound frame of theplurality of ultrasound frames against an artificial intelligence (“AI”)model to predict a different cut line on the ultrasound frame, whereineach cut line is positioned exterior to an image of the fetus appearingon the ultrasound frame; using the different cut lines on the pluralityof ultrasound frames to generate the 3D representation of the fetus.

In some embodiments, the method is performed without manual input of aregion of interest on any of the acquired plurality of ultrasoundframes.

In some embodiments, the AI model is trained with a plurality oftraining ultrasound frames each having a labeled cut line that ispositioned relative to an imaged fetus in the training ultrasound frame.

In some embodiments, at least some of the labeled cut lines are definedusing manual input.

In some embodiments, a cut line on the training ultrasound frame islabeled as acceptable if all points on the cut line are positionedexterior to the imaged fetus in the training ultrasound frame and allpoints on the cut line are positioned exterior to an imaged placenta,uterine wall, amniotic sac, umbilical cord, cervix, and bladder.

In some embodiments, a cut line on the training ultrasound frame islabeled as acceptable if all points on the cut line are positionedwithin amniotic fluid imaged in the training ultrasound frame.

In some embodiments, a cut line on the training ultrasound frame islabeled as unacceptable if any point on the cut line is positioned on orinterior to the imaged fetus in the training ultrasound frame or anypoint on the cut line is positioned on or interior to an imagedplacenta, uterine wall, amniotic sac, umbilical cord, cervix, orbladder.

In some embodiments, when using the different cut lines on the pluralityof ultrasound frames to generate the 3D representation of the fetus, themethod further comprises: removing, from each of the ultrasound frames,ultrasound data on a distal side of the cut line relative to the imageof the fetus appearing on the ultrasound frame; and using ultrasounddata that remains in each of the plurality of ultrasound frames togenerate the 3D representation of the fetus.

In some embodiments, when using the different cut lines on the pluralityof ultrasound frames to generate the 3D representation of the fetus, themethod further comprises using ultrasound data on a proximal side of thecut line relative to the image of the fetus appearing on the ultrasoundframe to generate the 3D representation.

In some embodiments, the method comprises determining, during theprocessing step, a confidence level for each of the predicted cut lines;wherein when using the different cut lines on the plurality ofultrasound frames to generate the 3D representation of the fetus, themethod further comprises: removing, from each of the plurality ofultrasound frames for which the confidence level is above a threshold,ultrasound data on a distal side of the cut line relative to the imageof the fetus appearing on the ultrasound data frame; and using remainingultrasound data in each of the ultrasound frames from which ultrasounddata has been removed to generate the 3D representation.

In some embodiments, the method comprises repeating the acquiring,processing, and using steps for subsequently acquired ultrasound framesof the fetus, wherein the 3D representation is projected onto a realtime image.

In some embodiments, the predicted cut lines on the plurality ofultrasound frames are each positioned within imaged amniotic fluid thatsurrounds the fetus.

In some embodiments, the predicted cut lines on the plurality ofultrasound frames are each positioned interior to an imaged uterus thatsurrounds the fetus.

In some embodiments, the predicted cut lines each comprise a Béziercurve, a polynomial curve, a spline curve, a freeform curve, aparametric curve, or a selection of two or more therefrom.

In some embodiments, the plurality of ultrasound frames of the fetuscontain imaged fetal anatomy, the fetal anatomy comprising a face, ahead, an ear, a nose, an eye, a neck, a torso, a foot, a leg, a hand, anarm, or a selection of two or more therefrom.

In some embodiments, the fetus is positioned adjacent to a wall of anamniotic sac, and the predicted cut lines on the plurality of ultrasoundframes delineate the fetus from the wall of the amniotic sac.

In some embodiments, when using the different cut lines on the pluralityof ultrasound frames to generate the 3D representation of the fetus, themethod further comprises removing, from one or more of the ultrasoundframes, data representing an imaged umbilical cord.

In some embodiments, the ultrasound scanner is held steadily in thearbitrary orientation by an operator of the ultrasound scanner while amotor in the ultrasound scanner tilts a transducer in the ultrasoundscanner so that each of the plurality of ultrasound frames is acquiredat a different angle of the transducer.

In some embodiments, the method comprises labeling each of the predictedcut lines as either acceptable or unacceptable, with at least one of thepredicted cut lines being labeled as unacceptable; and using theultrasound frames with labeled cut lines for training or reinforcing theAI model.

In some embodiments, the method comprises determining, by a processor, aphysical dimension of the fetus using the 3D model of the fetus; andcalculating an age of the fetus using the physical dimension.

In some embodiments, the method is performed without human input tomeasure the physical dimension.

In some embodiments, the physical dimension comprises one or more of:crown rump length (CRL), biparietal diameter (BPD), head circumference(HC), abdominal circumference (AC), femur length (FL), cephalic index(CI), humerus length (HL), Binocular Distance (BOD), or IntraocularDiameter (IOD).

Also disclosed herein is an ultrasound system for generating a threedimensional (“3D”) representation of a fetus, the system comprising anultrasound scanner configured to acquire a plurality of ultrasoundframes of the fetus with the ultrasound scanner in an arbitraryorientation with respect to the fetus; and a processor configured to:process each ultrasound frame of the plurality of ultrasound framesagainst an artificial intelligence (“AI”) model to predict a differentcut line on the ultrasound frame, wherein each cut line is positionedexterior to an image of the fetus appearing on the ultrasound frame; anduse the different cut lines on the plurality of ultrasound frames togenerate the 3D representation of the fetus.

In some embodiments, the processor is configured to generate the 3Drepresentation of the fetus without manual input of a region of intereston any of the acquired plurality of ultrasound frames.

In some embodiments, the AI model is trained with a plurality oftraining ultrasound frames each having a labeled cut line that ispositioned relative to an imaged fetus in the training ultrasound frame.

In some embodiments, at least some of the labeled cut lines are definedusing manual input.

In some embodiments, a cut line on the training ultrasound frame islabeled as acceptable if all points on the cut line are positionedexterior to the imaged fetus in the training ultrasound frame and allpoints on the cut line are positioned exterior to an imaged placenta,uterine wall, amniotic sac, umbilical cord, cervix, and bladder.

In some embodiments, a cut line on the training ultrasound frame islabeled as acceptable if all points on the cut line are positionedwithin amniotic fluid imaged in the training ultrasound frame.

In some embodiments, a cut line on the training ultrasound frame islabeled as unacceptable if any point on the cut line is positioned on orinterior to the imaged fetus in the training ultrasound frame or anypoint on the cut line is positioned on or interior to an imagedplacenta, uterine wall, amniotic sac, umbilical cord, cervix, orbladder.

In some embodiments, when using the different cut lines on the pluralityof ultrasound frames to generate the 3D representation of the fetus, theprocessor is further configured to remove, from each of the ultrasoundframes, ultrasound data on a distal side of the cut line relative to theimage of the fetus appearing on the ultrasound frame; and use ultrasounddata that remains in each of the plurality of ultrasound frames togenerate the 3D representation of the fetus.

In some embodiments, when using the different cut lines on the pluralityof ultrasound frames to generate the 3D representation of the fetus, theprocessor is further configured to use ultrasound data on a proximalside of the cut line relative to the image of the fetus appearing on theultrasound frame to generate the 3D representation.

In some embodiments, the processor is further configured to determine,during the processing step, a confidence level for each of the predictedcut lines; and wherein when using the different cut lines on theplurality of ultrasound frames to generate the 3D representation of thefetus, the processor is further configured to: remove, from each of theplurality of ultrasound frames for which the confidence level is above athreshold, ultrasound data on a distal side of the cut line relative tothe image of the fetus appearing on the ultrasound data frame; and useremaining ultrasound data in each of the ultrasound frames from whichultrasound data has been removed to generate the 3D representation.

In some embodiments, the ultrasound scanner and processor are furtherconfigured to repeatedly generate the 3D representation and project itonto a real time image.

In some embodiments, the predicted cut lines on the plurality ofultrasound frames are each positioned within imaged amniotic fluid thatsurrounds the fetus.

In some embodiments, the predicted cut lines on the plurality ofultrasound frames are each positioned interior to an imaged uterus thatsurrounds the fetus.

In some embodiments, the predicted cut lines each comprise a Béziercurve, a polynomial curve, a spline curve, a freeform curve, aparametric curve, or a selection of two or more therefrom.

In some embodiments, the plurality of ultrasound frames of the fetuscontain imaged fetal anatomy, the fetal anatomy comprising a face, ahead, an ear, a nose, an eye, a neck, a torso, a foot, a leg, a hand, anarm, or a selection of two or more therefrom.

In some embodiments, the fetus is positioned adjacent to a wall of anamniotic sac, and the predicted cut lines on the plurality of ultrasoundframes delineate the fetus from the wall of the amniotic sac.

In some embodiments, when using the different cut lines on the pluralityof ultrasound frames to generate the 3D representation of the fetus, theprocessor is further configured to remove, from one or more of theultrasound frames, data representing an imaged umbilical cord.

In some embodiments, the ultrasound scanner comprises a motor in theultrasound scanner that tilts a transducer in the ultrasound scanner sothat each of the plurality of ultrasound frames is acquired at adifferent angle of the transducer, while the ultrasound scanner is heldsteadily in the arbitrary orientation by an operator of the ultrasoundscanner.

In some embodiments, the processor is further configured to label eachof the predicted cut lines as either acceptable or unacceptable, with atleast one of the predicted cut lines being labeled as unacceptable; anduse the ultrasound frames with labeled cut lines for training orreinforcing the AI model.

In some embodiments, the processor is further configured to determine aphysical dimension of the fetus using the 3D model of the fetus; andcalculate an age of the fetus using the physical dimension.

In some embodiments, the processor is further configured to determinethe physical dimension and calculate the age without human input tomeasure the physical dimension.

In some embodiments, the physical dimension comprises one or more of:crown rump length (CRL), biparietal diameter (BPD), head circumference(HC), abdominal circumference (AC), femur length (FL), cephalic index(CI), humerus length (HL), Binocular Distance (BOD), or IntraocularDiameter (IOD).

Still further disclosed is a non-transient computer-readable mediastoring computer-readable instructions, which, when executed by aprocessor cause the processor to receive a plurality of ultrasoundframes of a fetus from an ultrasound scanner in an arbitrary orientationwith respect to the fetus; process each ultrasound frame of theplurality of ultrasound frames against an artificial intelligence (“AI”)model to predict a different cut line on the ultrasound frame, whereineach cut line is positioned exterior to an image of the fetus appearingon the ultrasound frame; and use the different cut lines on theplurality of ultrasound frames to generate a 3D representation of thefetus.

In some embodiments, the computer-readable instructions further causethe processor to generate the 3D representation of the fetus withoutmanual input of a region of interest on any of the acquired plurality ofultrasound frames.

In some embodiments, the AI model is trained with a plurality oftraining ultrasound frames each having a labeled cut line that ispositioned relative to an imaged fetus in the training ultrasound frame.

In some embodiments, at least some of the labeled cut lines are definedusing manual input.

In some embodiments, a cut line on the training ultrasound frame islabeled as acceptable if all points on the cut line are positionedexterior to the imaged fetus in the training ultrasound frame and allpoints on the cut line are positioned exterior to an imaged placenta,uterine wall, amniotic sac, umbilical cord, cervix, and bladder.

In some embodiments, a cut line on the training ultrasound frame islabeled as acceptable if all points on the cut line are positionedwithin amniotic fluid imaged in the training ultrasound frame.

In some embodiments, a cut line on the training ultrasound frame islabeled as unacceptable if any point on the cut line is positioned on orinterior to the imaged fetus in the training ultrasound frame or anypoint on the cut line is positioned on or interior to an imagedplacenta, uterine wall, amniotic sac, umbilical cord, cervix, orbladder.

In some embodiments, when using the different cut lines on the pluralityof ultrasound frames to generate the 3D representation of the fetus, thecomputer-readable instructions further cause the processor to: remove,from each of the ultrasound frames, ultrasound data on a distal side ofthe cut line relative to the image of the fetus appearing on theultrasound frame; and use ultrasound data that remains in each of theplurality of ultrasound frames to generate the 3D representation of thefetus.

In some embodiments, when using the different cut lines on the pluralityof ultrasound frames to generate the 3D representation of the fetus, thecomputer-readable instructions further cause the processor to: useultrasound data on a proximal side of the cut line relative to the imageof the fetus appearing on the ultrasound frame to generate the 3Drepresentation.

In some embodiments, the computer-readable instructions further causethe processor to: determine, during the processing step, a confidencelevel for each of the predicted cut lines; and wherein when using thedifferent cut lines on the plurality of ultrasound frames to generatethe 3D representation of the fetus, cause the processor to: remove, fromeach of the plurality of ultrasound frames for which the confidencelevel is above a threshold, ultrasound data on a distal side of the cutline relative to the image of the fetus appearing on the ultrasound dataframe; and use remaining ultrasound data in each of the ultrasoundframes from which ultrasound data has been removed to generate the 3Drepresentation.

In some embodiments, the computer-readable instructions further causethe processor to repeatedly generate the 3D representation and projectit onto a real time image.

In some embodiments, the predicted cut lines on the plurality ofultrasound frames are each positioned within imaged amniotic fluid thatsurrounds the fetus.

In some embodiments, the predicted cut lines on the plurality ofultrasound frames are each positioned interior to an imaged uterus thatsurrounds the fetus.

In some embodiments, the predicted cut lines each comprise a Béziercurve, a polynomial curve, a spline curve, a freeform curve, aparametric curve, or a selection of two or more therefrom.

In some embodiments, the plurality of ultrasound frames of the fetuscontain imaged fetal anatomy, the fetal anatomy comprising a face, ahead, an ear, a nose, an eye, a neck, a torso, a foot, a leg, a hand, anarm, or a selection of two or more therefrom.

In some embodiments, the fetus is positioned adjacent to a wall of anamniotic sac, and the predicted cut lines on the plurality of ultrasoundframes delineate the fetus from the wall of the amniotic sac.

In some embodiments, when using the different cut lines on the pluralityof ultrasound frames to generate the 3D representation of the fetus, thecomputer-readable instructions further cause the processor to remove,from one or more of the ultrasound frames, data representing an imagedumbilical cord.

In some embodiments, the computer-readable instructions further causethe processor to control a motor in the ultrasound scanner to tilts atransducer in the ultrasound scanner so that each of the plurality ofultrasound frames is acquired at a different angle of the transducer,while the ultrasound scanner is held steadily in the arbitraryorientation by an operator of the ultrasound scanner.

In some embodiments, the computer-readable instructions further causethe processor to: label each of the predicted cut lines as eitheracceptable or unacceptable, with at least one of the predicted cut linesbeing labeled as unacceptable; and use the ultrasound frames withlabeled cut lines for training or reinforcing the AI model.

In some embodiments, the computer-readable instructions further causethe processor to: determine a physical dimension of the fetus using the3D model of the fetus; and calculate an age of the fetus using thephysical dimension.

In some embodiments, the computer-readable instructions further causethe processor to determine the physical dimension and calculate the agewithout human input to measure the physical dimension.

In some embodiments, the physical dimension comprises one or more of:crown rump length (CRL), biparietal diameter (BPD), head circumference(HC), abdominal circumference (AC), femur length (FL), cephalic index(CI), humerus length (HL), Binocular Distance (BOD), or IntraocularDiameter (IOD).

The invention claimed is:
 1. A method for defining cut lines to generatea three-dimensional (“3D”) representation of a fetus, the methodcomprising: acquiring a plurality of two-dimensional (“2D”) ultrasoundframes of the fetus using an ultrasound scanner in an arbitraryorientation with respect to the fetus; processing, by a processor, each2D acquired ultrasound frame of the plurality of 2D acquired ultrasoundframes against an artificial intelligence (“AI”) model, the AI Modelbeing trained with a plurality of 2D training ultrasound frames eachframe being on a plane, as scanned, and each having a training cut linethereon, wherein a training cut line on a 2D training ultrasound frameis labeled as acceptable if all points on the training cut line arepositioned within amniotic fluid imaged in the plane of the 2D trainingultrasound frame, wherein the processing of the 2D acquired ultrasoundframe against the AI model is to predict a cut line (predicted cut line)on each of the 2D acquired ultrasound frames; and using the differentpredicted cut lines on the plurality of 2D acquired ultrasound frames togenerate the 3D representation of the fetus.
 2. The method of claim 1,wherein the AI model is trained with a further plurality of 2D trainingultrasound frames and each 2D training ultrasound frame of the furtherplurality of 2D training ultrasound frames has a labeled training cutline that is positioned relative to an imaged fetus in the 2D trainingultrasound frame, the imaged fetus being different from the fetus in theplurality of 2D acquired ultrasound frames.
 3. The method of claim 1,wherein a training cut line on each 2D training ultrasound frame of thefurther plurality of 2D training ultrasound frames is labeled asacceptable if all points on the training cut line are positionedexterior to the imaged fetus in the 2D training ultrasound frame and allpoints on the training cut line are positioned exterior to an imagedplacenta, uterine wall, amniotic sac, umbilical cord, cervix, andbladder.
 4. The method of claim 1, wherein when using the differentpredicted cut lines on the plurality of 2D acquired ultrasound frames togenerate the 3D representation of the fetus, the method furthercomprises: removing, from each of the 2D acquired ultrasound frames,ultrasound data on a distal side of the cut line relative to the imageof the fetus appearing on the 2D acquired ultrasound frame; and usingultrasound data that remains in each of the plurality of 2D acquiredultrasound frames to generate the 3D representation of the fetus.
 5. Themethod of claim 1, wherein when using the different predicted cut lineson the plurality of 2D acquired ultrasound frames to generate the 3Drepresentation of the fetus, the method further comprises: usingultrasound data on a proximal side of the cut line relative to the imageof the fetus appearing on the 2D acquired ultrasound frame to generatethe 3D representation of the fetus.
 6. The method of claim 1, furthercomprising: determining, during the processing step, a confidence levelfor each of the predicted cut lines; and wherein when using thedifferent predicted cut lines on the plurality of 2D acquired ultrasoundframes to generate the 3D representation of the fetus, the methodfurther comprises: removing, from each of the plurality of 2D acquiredultrasound frames for which the confidence level is above a threshold,ultrasound data on a distal side of the cut line relative to the imageof the fetus appearing on the 2D acquired ultrasound frame; and usingremaining ultrasound data in each of the 2D acquired ultrasound framesfrom which ultrasound data has been removed to generate the 3Drepresentation of the fetus.
 7. The method of claim 1, comprising:repeating the acquiring, processing, and using steps for subsequentlyacquired 2D acquired ultrasound frames of the fetus, wherein the 3Drepresentation is projected onto a real time image.
 8. The method ofclaim 1, wherein the predicted cut lines each comprise a Bézier curve, apolynomial curve, a spline curve, a freeform curve, a parametric curve,or a selection of two or more therefrom.
 9. The method of claim 1,wherein the fetus is positioned adjacent to a wall of an amniotic sac,and the predicted cut lines on the plurality of 2D acquired ultrasoundframes delineate the fetus from the wall of the amniotic sac.
 10. Themethod of claim 1, wherein the ultrasound scanner is held steadily inthe arbitrary orientation by an operator of the ultrasound scanner whilea motor in the ultrasound scanner tilts a transducer in the ultrasoundscanner so that each of the plurality of 2D acquired ultrasound framesis acquired at a different angle of the transducer.
 11. An ultrasoundsystem for generating a three-dimensional (“3D”) representation of afetus, the system comprising: an ultrasound scanner configured toacquire a plurality of two-dimensional (“2D”) acquired ultrasound framesof the fetus with the ultrasound scanner in an arbitrary orientationwith respect to the fetus; and a processor configured to: process each2D acquired ultrasound frame of the plurality of 2D acquired ultrasoundframes against an artificial intelligence (“AI”) model, the AI modelbeing trained with a plurality of 2D training ultrasound frames eachframe being on a plane, as scanned, and each having a training cut linethereon, wherein a 2D training ultrasound frame wherein a training cutline on a 2D training ultrasound frame is labeled as acceptable if allpoints on the training cut line are positioned within amniotic fluidimaged in the plane of the 2D training ultrasound frame, wherein theprocessing of the 2D acquired ultrasound frame against the AI model isto predict a cut line (predicted cut line) on each of the 2D acquiredultrasound frames; and use the different predicted cut lines on theplurality of 2D acquired ultrasound frames to generate the 3Drepresentation of the fetus.
 12. The ultrasound system of claim 11,wherein the AI model is trained with a further plurality of 2D trainingultrasound frames and each 2D training ultrasound frame of the furtherplurality of 2D training ultrasound frames has a labeled training cutline that is positioned relative to an imaged fetus in the 2D trainingultrasound frame, the imaged fetus being different from the fetus in theacquired plurality of 2D acquired ultrasound frames.
 13. The ultrasoundsystem of claim 11, wherein a training cut line on each 2D trainingultrasound frame of the further plurality of 2D training ultrasoundframes is labeled as acceptable if all points on the training cut lineare positioned exterior to the imaged fetus in the 2D trainingultrasound frame and all points on the training cut line are positionedexterior to an imaged placenta, uterine wall, amniotic sac, umbilicalcord, cervix, and bladder.
 14. The ultrasound system of claim 11,wherein when using the different predicted cut lines on the plurality of2D acquired ultrasound frames to generate the 3D representation of thefetus, the processor is further configured to: remove, from each of the2D acquired ultrasound frames, ultrasound data on a distal side of thecut line relative to the image of the fetus appearing on the 2D acquiredultrasound frame; and use ultrasound data that remains in each of theplurality of 2D acquired ultrasound frames to generate the 3Drepresentation of the fetus.
 15. The ultrasound system of claim 11,wherein when using the different predicted cut lines on the plurality of2D acquired ultrasound frames to generate the 3D representation of thefetus, the processor is further configured to: use ultrasound data on aproximal side of the cut line relative to the image of the fetusappearing on the 2D acquired ultrasound frame to generate the 3Drepresentation.
 16. The ultrasound system of claim 11, wherein theprocessor is further configured to: determine, during the processingstep, a confidence level for each of the predicted cut lines; andwherein when using the different predicted cut lines on the plurality of2D acquired ultrasound frames to generate the 3D representation of thefetus, the processor is further configured to: remove, from each of theplurality of 2D acquired ultrasound frames for which the confidencelevel is above a threshold, ultrasound data on a distal side of the cutline relative to the image of the fetus appearing on the 2D acquiredultrasound frame; and use remaining ultrasound data in each of the 2Dacquired ultrasound frames from which ultrasound data has been removedto generate the 3D representation.
 17. The ultrasound system of claim11, wherein the ultrasound scanner and processor are further configuredto: repeatedly generate the 3D representation and project it onto a realtime image.
 18. A non-transitory computer-readable media storingcomputer-readable instructions, which, when executed by a processorcause the processor to: receive a plurality of two-dimensional (“2D”)acquired ultrasound frames of a fetus from an ultrasound scanner in anarbitrary orientation with respect to the fetus; process each 2Dacquired ultrasound frame of the plurality of 2D acquired ultrasoundframes against an artificial intelligence (“AI”) model, the AI modelbeing trained with a plurality of 2D training ultrasound frames eachframe being on a plane, as scanned, and each having a training cut linethereon, wherein a training cut line on a 2D training ultrasound frameis labeled as acceptable if all points on the training cut line arepositioned within amniotic fluid imaged in the plane of the 2D trainingultrasound frame, wherein the processing of the 2D acquired ultrasoundframe against the AI model to predict a cut line (predicted cut line) oneach of the 2D acquired ultrasound frames; and use the differentpredicted cut lines on the plurality of 2D acquired ultrasound frames togenerate a three-dimensional (3D) representation of the fetus.
 19. Thenon-transitory computer-readable media of claim 18, wherein the AI modelis trained with a further plurality of 2D training ultrasound frames andeach 2D training ultrasound frame of the further plurality of 2Dtraining ultrasound frames has a labeled training cut line that ispositioned relative to an imaged fetus in the 2D training ultrasoundframe, the imaged fetus being different from the fetus in the acquiredplurality of 2D acquired ultrasound frames.
 20. The non-transitorycomputer-readable media of claim 19, wherein a training cut line on each2D training ultrasound frame of the further plurality of 2D trainingultrasound frames is labeled as acceptable if all points on the trainingcut line are positioned exterior to the imaged fetus in the 2D trainingultrasound frame and all points on the training cut line are positionedexterior to an imaged placenta, uterine wall, amniotic sac, umbilicalcord, cervix, and bladder.